Pascal Super Library/Pascal Super Library (CW International)(1997).bin/LIBRARY/SLTPU70C/SLTPU.DOC

Text File | 1993-09-15 | 19KB | 386 lines

                     Searchlight Programmer's Library
                               Documentation
                  (c) Copyright 1993 Searchlight Software

I. INTRODUCTION

Searchlight Programmer's Library is a series of Turbo Pascal unit (TPU)
files which can be linked into Turbo Pascal programs, providing a wide range
of functions for manipulating Searchlight data files. Included are
procedures to access, add and delete users, members, messages and files from
SLBBS data structures.

Units in this library provide low and midlevel file i/o operations
exclusively. No user input/output oriented procedures are provided.

It is not the intention of this documentation to describe the Searchlight
data structures at their lowest level. In particular, the details of binary
tree management, free record management, and subboard message indexing are
not covered. Instead, the library functions provided by the various TPU
files automatically implement these data structures and spare the
applications programmer from having to re-invent the low level code for
doing so. We do explain the workings of mid to high level data structures
necessary for writing complex programs that access the Searchlight database.

This file provides general principles and recommendations for using the TPU
library for various types of applications. For more information, consult:

SLTPU.REF    - Complete reference to provided procedures and functions.
FILEDEF.REF  - Record layouts provided by the FILEDEF.TPU unit.
BLOCK.REF    - Record layouts provided in the BLOCK.TPU unit.
STYLEDEF.REF - Record layouts for the STYLEDEF.TPU unit.
SAMPLE?.PAS  - Example programs.


II. UNIT INSTALLATION AND USEAGE

The TPU files contained in this development package are designed to work
with a particular version of Turbo Pascal. Before using these units you
must be sure that you have the correct version of the TPU files to match the
version of Turbo Pascal you will be using. Versions for the various
editions of Turbo Pascal are available on the Searchlight Support BBS (516-
689-2566).

The TPU files should be copied to your hard drive into either a working
directory where programs are to be developed, a directory containing other
TPU files or a separate directory. If the TPU files are not in the current
directory when you run Turbo Pascal, set Turbo's TPU path to point to the
directory where they are stored.

To use these units in your programs, simply declare the unit name in your
program's USES clause. See sample programs for examples.


2A. Existing Programs

Many of the TPU files in this library assume that you will be using the
global variables found in those units; for example, the POST unit works with
the global variable 'MainSub', which defines the current subboard.

Because of this situation, care should be taken when incorporating these
units into existing programs. In particular, existing low level code used
to manage data files should be discarded entirely in favor of direct calls
to the library procedures. Be aware of the global variables declared in
units such as FILEDEF, USERS and MEMBERS, and be sure that these do not
conflict with locally declared variables.

Wherever possible, we recommend building new applications from scratch
rather than converting existing programs.


2B. Other Languages

The TPU library is designed for use with Turbo Pascal. In order to get the
most out of these tools, we recommend you do all of your development with a
Turbo Pascal compiler.

If you must do development in another environment, here are some
recommendations:

(1) The functions found in MODEM.PAS for accessing Searchlight's serial port
drivers are basically 8086 assembly language function calls. These are
readily translated into other languages. The source code should be self-
explanatory.

(2) Searchlight's flat files, such as the CONFIG file, can be accessed from
any language simply by converting the Pascal record layout to the proper
structure, or by computing the byte offsets to the required variables. The
same technique will work for the in-memory variables used in MODEM.PAS.

It is NOT recommended that you attempt to write your own procedures for
accessing Searchlight's more complex data files, such as the user file or
message subboard files. These files use a complex system of indexes and
linked lists which the Pascal library code manages for you.

(3) If you wish to write your main application in another language but still
have access to the SLTPU functions, consider writing a separate program in
Turbo Pascal that can be accessed by your main program. For example, your
main program could call a separate EXE file written in Pascal when it needs
to perform a function such as saving a message. Or you could write a TSR
type of program that provides services via INT functions to your main
program.



III. READING MESSAGES

Each message in a Searchlight BBS subboard (which includes the Mail and
Bulletin subboards) consists of two basic elements: the message header, and
the message text. Headers are stored in the subboard's header (HDR) file,
and text in the message (MSG) file.

Message headers contain all of the information about a message other than
the text lines of the message itself. This includes such information as who
the message is from and who it's to; the date and time the message was
posted; and the message's subject. It also includes information about how
the message relates to other messages in the current subboard, ie.
information about the next and previous message (in both sequential and
threaded order), pointers to the message's replies and its parent, etc.
There is 1 header record (type 'HeaderType' as defined in FILEDEF.PAS) per
message.

Each message contains a pointer to the message's text in the MSG file. Text
consists of one or more records, depending on the size of the message. The
packing and unpacking of these blocks is automatically handled by the
library procedures; programmers need only be familiar with the in-memory
representation as described in the accompanying reference manual.

There are 3 ways to locate a message's header file (and thus the message
itself) on disk: by record number, by message number, or by an internal
number called the UID or Universal ID value.

Record number refers to the physical location of the header record in the
header file. If the record number of a particular header is know, it can be
retrieved simply by reading the record from disk.

Message number refers to the sequential number assigned to each message in a
subboard; this is the number you usually see when reading the message in
Searchlight BBS (however note that certain commands, notably the MAIL
command, do not display the actual message numbers of messages but instead
display sequential numbers for convenience).

UID values are 32 bit numbers computed from the date and time a message is
posted plus a 7 bit random value. UID's are designed to provide a universal
way of identifying any message created on any subboard or even on any BBS,
and should stay with the message when it is imported and exported.  Since
the thread pointers in Searchlight are keyed to UID values rather than
message numbers, threading can be maintained even when messages are exported
to other subboards or other BBS systems.

It is very important to make the distinction between message numbers, UID
numbers and record numbers. A record number is simply a physical location,
which may bear no relationship to the message number, UID number, age of the
message, or anything else. Message and UID numbers are a logical values
stored within the header of the message itself. When you know the message
number or UID of a given message and wish to retrieve its header, you can do
so only by searching for the header. Fortunately, Searchlight subboards
implement an index which makes such searching for either value very fast.
In this library, you can search via the index simply by using the
'FindHeader' command implemented in MESSAGE.TPU (see SLTPU.REF for more
information).


3A. Message Pointer Types

Message numbers, record numbers, and UID numbers are all stored as 32 bit
values in Searchlight (ie. LongInt values). To navagate a subboard
successfully, you need to know what kind of value each 32 bit pointer
represents.

Within the 'Headertype' variable, the pointers are:

       id: array[SEQ..UID]
         of longint;         { sequential and universal ID numbers }

These are the Message Number and UID values of the message itself; Id[Seq]
is a message number, and Id[Uid] is a UID number.

       lastseq,
       nextseq: longint;     { next/previous sequential message }

These values are message numbers. 'LastSeq' is the message number of the
previous message, and 'NextSeq' the number of the next message. If either
value is 0, it means that there is no last or next message. Notice that
since Searchlight does not renumber messages when a deletion occurs, the
next message number is not always equal to the current message number plus
1, nor is the last message always the current message minus 1. Therefore,
use these pointers to traverse the message base as a linked list.

       lastthread,
       nextthread: longint;  { next/previous threaded message }

All thread pointers are UID values: when you look up a message based on one
of these pointers, use the 'Uid' parameter to 'FindHeader'. The lastthread
and nextthread pointers form a linked list of messages in threaded order.
In order to follow a thread, simply follow this linked list instead of the
lastseq and nextseq pointers.

       topthread,            { first msg in this thread }
       lastreply,            { last reply to this message }
       replyto: longint;     { message to which this is a direct reply }

Again, these are UID values. 'Topthread' points to the first message in the
thread to which the current message belongs; 'Lastreply' points to the last
reply to the current message (0 if no replies) and 'Replyto' points to the
parent to this message (if the message is a reply).

All of the messages pointed to by these pointers will be part of the thread
formed by the 'lasttthread' and 'nextthread' pointers. You can use these
pointers primarily to define sub portions of a thread (such as the replies
to the current message) or to directly access special messages, such as the
message to which the current message replies.

       lastmail,
       nextmail: longint;    { next/previous personal mail message }

These are message numbers, and form a linked list pointing to all messages
in the current subboard which are addressed to the same user. In the MAIL
subboard, this linked list forms the user's mailbox and allows you to obtain
all the mail for a particular user without examining every message in the
subboard. The first mail message for a user (ie. the root of this list) is
stored in the user's Member record.

Notice that these pointers are active for all subboards, not just MAIL; they
can be used to obtain a list of messages addressed to a particular user on
any subboard.

       nextseqrec: longint;  { record # of next sequential msg }
       nextmailrec: longint; { record # of next mail message }

These values are the only two record number values in the message header,
and point to the record number of the next sequential message and the next
mail message (ie. the actual record numbers of the messages pointed to by
'Nextseq' and 'Nextmail').

The reason these two values are coded as record numbers is to allow speedy
processing of messages or MAIL in sequential order. When processing
headers, you can pass the value of 'NextSeqRec' or 'NextMailRec' as the last
parameter of FindHeader in order to speed up the search.

       (type Subtype)
       firstmsg,
       lastmsg: longint;       { first & last active msg by ID # }

These two message number values are stored in the header record of the
header file, and point to the first and last messages on the subboard.
These values are useful when you wish to begin processing an entire subboard
in forward or reverse order. Be sure to read the Subtype record from disk
right before using these values, as the values can change in multiuser
systems.

       (type MembType)
       lastread: longint;     { highest message read }
       firstmail,
       lastmail: longint;     { first/last personal message this subboard }

The member record for a particular user in the current subboard contains
information about the user's highest message read, and pointers to the first
and last message to that user in the subboard (on the MAIL subboard, these
pointers define the user's mailbox). These values are message numbers.


3B. New Message Maintenance

The highest message read by a particular user on a particular subboard can
be obtained by retrieving the user's member record from the subboard member
file. The 'LastRead' field within the member record indicates the highest
message read.

To begin reading the next highest message, add 1 to this number and attempt
to read the message header from disk. For example, if the high message
number for a particular message is 100, attempt to read message 101. If no
message 101 exists (ie. if the FindHeader procedure returns a result of 0),
then try to read message 102, etc. until you either find the message, or
the number you are searching for is higher than the highest message on the
subboard (Mainsub.Subinfo.Lastmsg).

Don't forget to update the member's high message pointer when you are
finished, if that is required.


3C. Mailbox Access

To access a user's mail on the MAIL subboard, or all messages to that user
on any subboard, begin with the 'FirstMail' value in the member record for
that user. If no member record exists, or if the FirstMail value is zero,
then no mail or personal messages are available.

After reading the first mail message, the next can be obtained by following
the 'Nextmail' or 'Nextmailrec' pointer in the message's header file.

On the MAIL subboard, the 'LastRead' field in the member file is not used to
indicate where new mail messages begin. Instead, you should determine which
messages are new by the fact that the number of times read ('Rd') value in
the header is zero. If you wish to mark a new message as having been read,
increment this value and update the header on disk.



IV. IMPORTING AND EXPORTING

A major category of useful programs involves the importing of messages in
various external formats to Searchlight BBS message areas, and the export of
Searchlight messages to external files. Examples of such programs include
Searchlight's SLMAIL program, which provides echomail capability, and the
UTI drivers, which provide compatibility with PCRelay and Megamail offline
reader software.

The following discussion will be of use to anyone writing a program of this
type, including programs designed to allow Searchlight BBS systems to
communicate with network mail systems, offline reader software, or other
similar projects.


4A. Message Exporting

When exporting messages to an external file, try to preserve as much of the
header information as possible. Most formats include space for the message
'To' and 'From' fields, as well as the subject.

If possible, try to preserve the outgoing message's UID number
(Header.Id[Uid]) and, if it is a reply, the UID number of the original
message (Header.ReplyTo). This will allow programs which import the message
into SLBBS systems to preserve the original threading associated with the
message.

Many external message formats have a field for "Reference Number", which can
be used to contain the 'ReplyTo' link if the field is large enough to hold a
4 byte integer value. In offline reader programs, you can store the message
number of the original message (as opposed to its UID value). This will
also allow threading to be maintained, but only if replies are imported into
the same subboard on the same BBS from which the original message was
exported. It is preferable to use UID numbers wherever possible, as these
will allow threading to be maintained even if messages are re-imported into
a different subboard.

When working with established data formats which do not provide an explicit
field for message UID number and reference (ReplyTo) number, try to encode
this information as part of the message text (for example, as a hidden line
if that is supported) or in an unused header field. Of course, you should
consult with the authors of the data format before using any fields which
might cause a problem for existing programs which process that format.

Most exportation programs will require a means to save the number of the
last message exported from a subboard, so that exportation of new messages
can begin at the next highest message. You can do this either externally in
your own file, or internally by creating a MEMBER record for your program in
the subboard's member file and updating the last read pointer there.


4B. Message Importing

When importing data from an external file format, try to determine whether
the message contains a Searchlight UID number and 'ReplyTo' UID value. If
these can be obtained from the external message, load the UID value into the
message header (Header.Id[Uid]) before saving the message and pass FALSE for
the 'SetUid' parameter of the 'MsgPost' function. Pass the ReplyTo value as
the 'Original' parameter to MsgPost.

If you know the message number of the original message instead of the UID
value, you can pass this value times -1 to the MsgPost procedure. This will
work only if you are importing a reply to the same subboard in which the
original message was located (for example, in an offline reader system) and
no renumbering has occured. Do not attempt to use this method if you know
that messages may be imported into a different subboard (for example, in a
network mail situation).

If no message UID can be found in the external file, pass TRUE for 'SetUid'.
If no reference or replyto value can be found, you can either pass 0 for the
original message value, or, if it is customary for the system in which you
are working, try to determine the ID of the original message by comparing
the subject field of the message with other messages on the current
subboard.

Searchlight expects messages to contain a maximum of 400 lines with a
maximum line length of 76 characters per line. If the incoming message
contains lines longer than 76 characters or if it is longer than 400 lines,
the message may need to be reformatted and/or truncated in order to fit
within Searchlight's boundaries. Message lines should not contain control
characters (ASCII values less than 32) but may contain extended IBM graphics
characters (ASCII values up to 255).


(c) Copyright 1991-1993 Searchlight Software